Automatic Selection of Split Criterion during Tree Growing Based on Node Location

Typically, decision tree construction algorithms apply a single \goodness of split" criterion to form each test node of the tree. It is a hypothesis of this research that better results can be obtained if during tree construction one applies a split criterion suited to the \location" of the test node in the tree. Speciically, given the objective of maximizing predictive accuracy, test nodes near the root of the tree should be chosen using a measure based on information theory, whereas test nodes closer to the leaves of the pruned tree should be chosen to maximize classii-cation accuracy on the training set. The results of an empirical evaluation illustrate that adapting the split criterion to node location can improve classiication performance. 1 DECISION TREE CONSTRUCTION A decision tree is either a leaf node containing a classi-cation or an attribute test, with for each value of the attribute, a branch to a decision tree. To classify an instance using a decision tree, one starts at the root node and nds the branch corresponding to the value of the test attribute observed in the instance. This process repeats at the subtree rooted at that branch until a leaf node is reached. The resulting classiication is the class label of the leaf. The objective of a decision tree construction algorithm is to create a tree such that the classiication accuracy of the tree when applied to previously unobserved instances is maximized (hereafter the predictive accuracy). Other criteria such as tree size and tree under-standability may also be of interest. For domains in which the cost of misclassifying instances is not uniform , the objective is to nd the tree such that misclas-siication cost (given by a misclassiication cost matrix) for previously unobserved instances is minimized. One well-known approach to constructing a decision tree is to grow a tree until each of the terminal nodes (leaves) contain instances from a single class and then prune back the tree with the goal of nding the sub-tree with the lowest misclassiication rate on previously unobserved instances (Breiman, Friedman, Olshen & Stone, 1984; Quinlan, 1986). During tree growing, at each node, one wants to select a test that best divides the instances into their classes. There are many diier-ent criteria that can be used to judge the \goodness of a split"; the most common appear in the form of an entropy or impurity measure. and compare splitting criteria. …